Method and System for Telecommunication with the Aid of Virtual Control Representatives

ABSTRACT

Method and system of telecommunication between at least two users over a telecommunications network, wherein the first user is connected to the telecommunications network via a first terminal and the second user via a second terminal, and wherein a virtual representative is allocated to each user. A first virtual representative is allocated to the first user and a second virtual representative to the second user. The first and second virtual representatives are presented on the first terminal and on the second terminal. Information is transferred from the first user to the second user and vice versa by an animation of at least one of the first and second representatives and by an interaction between the first and second representatives, wherein at least one of the animation and the interaction takes place in response to a drag &amp; drop command of a user, and wherein an animation of the first virtual representative takes place in response to a command of the first user and an animation of the second virtual representative takes place in response to a command of the second user.

The present invention relates to a method and a system by means of whichat least two users can communicate with one another via appropriateterminals. Communication is broadened and supported through the use ofvirtual representatives.

BACKGROUND

In addition to more conventional means of communication such astelephone, fax or e-mail there has for some time been a furthercommunications service which has become known as “Instant Messaging”(IM). With this communications service several users using a clientprogram can exchange written messages in real time or also “chat” withone another. A text inputted by a user is sent in real time to the otherparticipating users. These can then respond in turn to sent messages bytext input.

A disadvantage of Instant Messaging is that this form of communicationis limited to the exchange of pure text messages. In order to overcomethis disadvantage and to broaden the possibilities of expression inInstant Messaging, many users use so-called “emoticons”. Emoticons arecharacter strings imitating a face or also “smiley” which are used inwritten electronic communication to express moods and feelings.

Even if the possibilities of expression within Instant Messaging can beslightly broadened thanks to “emoticons” there is still no furtherpossibility here of communicating in particular emotions and moods to achat partner in a multi-layered, clear, attractive and multi-media way.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a method and a systemwith which at least two users of telecommunications terminals cancommunicate with one another in real time in a multi-layered, attractiveand multi-media way. The method according to the invention and thesystem according to the invention are in particular intended to makepossible a particularly direct, versatile and varied communication ofmoods, emotions and feelings.

The present invention provides a method of telecommunication between atleast two users over a telecommunications network wherein the first useris connected to the telecommunications network via a first terminal andthe second user via a second terminal, and wherein a virtualrepresentative is allocated to each user, with the following steps:

-   -   presentation of the two representatives on the first terminal        and on the second terminal;    -   transfer of information from the first user to the second user        and vice versa by an animation of at least one representative        and by an interaction between the representatives.

For the sake of clarity, in the following the case of a merely two-waycommunication between a first and a second user is always described.However, the invention naturally also covers appropriately designedcommunications between three or even more users.

In the method according to the invention, communication between the twousers is substantially broadened and improved through the use of virtualrepresentatives. The users are now no longer tied exclusively to thewritten form for exchanging information, but can also immediately passon information to the respective communications partner in vision andsound by animating their respective representative. The virtualrepresentative represents not only the respective user, but alsocomprises communications functions, in particular the functionsdescribed below for a non-verbal communication. Thus each representativeis not only to be understood as a graphic element, but also as a programelement or object for an application program which runs on the terminalof the respective user for the purpose of communication with the otheruser. The representatives are thus small communications controlprograms. The representatives are therefore also called “communicationsrobots”, “combots” for short, in the following.

By telecommunication is meant in the context of the inventioncommunication between the two users over a certain distance, and verybroadly understood. This means that all types of communication over allcommunications networks are included. This takes place for example overa telecommunications network, which can be for example a telephonenetwork, a radio communications network, a computer network, a satellitenetwork or a combination of these networks. The network according to theinvention preferably includes the Internet or else the World Wide Web.

Both the first user and also the second user communicate with each othervia so-called terminals. These terminals serve for telecommunication andmake possible the exchange of information between the users in vision,sound and/or in written form. These terminals can be telephones, mobilephones, PDAs, computers or similar. The users can also communicate withone another via different devices in each case. The terminals arepreferably Internet-capable computers or also PCs.

By “user” is meant in the context of the invention a natural person orelse a human individual.

According to the invention a virtual representative (combot) isallocated to each user when telecommunication takes place. This virtualrepresentative can also be called a doppelgänger or avatar. This is agraphic dummy which represents the respective user. For example, a usercan have a known comic figure such as Donald Duck or Bart Simpson asvirtual representative. The graphic figure is presented to the user onhis terminal during the communication. Simultaneously, the user alsosees a further graphic object, which stands for his communicationspartner.

Thus information, such as e.g. an expression of feeling, can becommunicated in a novel way to a communications partner by animating thevirtual representative of the communicating party accordingly.Additionally or alternatively an interaction between the tworepresentatives can also be presented.

If a representative is animated this means that the appearance or soundof its graphic presentation changes with time. A representative is thusnot merely a static picture or symbol, but is dynamic and can performthe most varied acts. Thus a representative can e.g. wave to show thegreeting.

If an interaction takes place between two representatives this meansthat not only each representative is animated independently of theother. Rather, one representative reacts to the action of the otherrepresentative and vice versa. Thus an interactive transaction takesplace between the two representatives, the two representatives influenceeach other and enter into a reciprocal relationship. Thus e.g. onerepresentative can wave back in response to a wave from the otherrepresentative.

The animation and/or interaction of the representative preferably takesplace in response to a user command, in particular in response to a drag& drop command from the user. Thus the user can control hisrepresentative individually in order to e.g. indicate his current moodto his communications partner by controlling the representative. Controltakes place by a corresponding operation of the respective terminal,which is preferably a personal computer. If the terminal has a graphicuser interface (desktop) with a mouse-type control the user canparticularly easily by dragging and dropping (drag & drop), trigger ananimation or interaction of his representative. For this, the user moveshis mouse pointer onto a graphic image of the animation which hisrepresentative is to carry out and “drags” this image onto the graphicpresentation of his representative. A predefined area of the desktop ora window or window area created by the application program can serve forthis.

An animation of the representative of the second user can preferablyalso take place in response to a command from the first user and viceversa. The described interaction between the representatives is thuseasily possible with this function. This function is useful inparticular if one user wishes his representative to carry out an actionwhich is to have an effect on the representative of the other user. Thusthe first user can e.g. instruct his representative to throw an item atthe representative of the other user. In response to the throw commandfrom the first user and the graphic presentation of the throw at therepresentative of the second user, there is an appropriate “reaction” ofthe representative at whom it has been thrown in the form of anappropriate animation. An animation of the representative of the seconduser is thus triggered by a control command from the first user. Thus akind of video or computer game can even develop between the two usersusing the two representatives. Preferably the first user can obtain suchan animation of the representative of the second user by the describeddrag & drop onto the representative of the second user.

The animation and/or interaction taking place in response to a usercommand is preferably presented simultaneously, parallel and in realtime on both terminals of the two users. This means that both users canfollow the behaviour of the representatives in response to the inputtedcommands, live, as it were, on their respective terminals.

Depending on how quickly and directly the exchange via therepresentatives is to take place, the control commands inputted by theusers to animate the representatives of the users can be processeddifferently. Thus a newly inputted user command can lead to a directinterruption of an ongoing animation or interaction; the interruption isthen followed immediately by the new animation desired by the user.Instead of this the ongoing animation or interaction can also becompleted first in response to a new user input, so that the desiredanimation follows on immediately from the completed animation.Furthermore when there are several user commands following in quicksuccession on both sides under certain circumstances the desiredanimations or interactions can be placed in a waiting list of theanimations or interactions to be carried out. The animations indicatedby the users are then processed in sequence according to the waitinglist.

The interruption to a first animation or interaction triggered by thefirst user and a replacement of the first animation or interaction withan animation or interaction triggered by the second user and vice versacan also take place. If, for example, the first user triggers aninteraction by which his representative fires an arrow at therepresentative of the second user, the second user could interrupt thisinteraction by instructing his representative to ward off the arrow witha shield. The first user could in turn interrupt this second interactionby triggering a further interaction and so on. Thus a regularinteractive game of action and reaction can develop between the twousers using the representatives.

The progress of the interaction can depend on predeterminable parameterswhich the users predetermine and/or are stored in the system in userprofiles which are allocated to the users. The parameters can includee.g. personal information about the respective user, such as, say, hisnationality, his place of residence or temporary location, hispreferences and hobbies etc. Thus e.g. idiosyncrasies in communication,in particular gestures, can be taken into account which are specific tothe respective nationality or culture group. Also, by means of dataacquisition and statistical functions, the respective user profile canbe managed by the system and brought up to date, so that the appropriateinteractions are automatically used for the representative (combot) ofthe respective user or at least an appropriate selection is offered tothe user, e.g. a number of the preferred interactions (favourites). Thesystem thus has at its disposal a function which automatically changesand adapts the interactions using the parameters. The user can switchthis auto-function on and off at any time.

According to a further independent inventive aspect, to further broadenthe depth of communication between the users, a recognition of a speechor text input into his terminal by a user can also take place. Therecognized speech or text input is then analyzed, so that its importanceis detected.

Furthermore a video recognition (e.g. by means of a video camera) of thefacial expression of a user and its analysis and interpretation can takeplace. Thus the facial expressions of a user can preferably be recordedand assessed for specific expressions of feeling.

Subsequent to the analysis and interpretation several suitablepossibilities for animation or interaction can be provided to the userin tune with the sense of his speech or text input or his facialexpression. If the user thus makes it known e.g. in writing, verbally orthrough his facial expression, that he is happy, appropriate animationsexpressing happiness (the animation “smile” or “laugh” or “jump” etc. .. . ) can be proposed to the user for the appropriate animation of hisrepresentative.

Instead of a proposal function, an animation of a representative and/oran interaction between the representatives in tune with the sense of thespeech or text input or the facial expression can also take placedirectly or automatically. In this case the sense of a speech or textmessage or the facial expression can be automatically established, andconsequently the behaviour of the corresponding representative canlikewise automatically be matched to the sense of the speech or textmessage or of the facial expression. If the speech or text message orfacial expression of a user thus says e.g. “I am sad” the representativeof the user can automatically adopt a sad facial expression.Alternatively there can be a confirmation by the user first before therepresentative imitates the recognized sense. The automatic recognitionof the sense of a text message can also be called “parsing”. The text issearched for keywords and terms for which appropriate animations andwhere appropriate interactions are offered to the user and/orautomatically introduced by the system into the communication. Such a“parsing” function can also be applied appropriately to non-textmessages, in particular to speech messages. Moreover, during analysis ofthe contents of the messages, information about the user can also beused which is retrieved from the user profile stored in the system. Thusinformation about writing and speech habits of the respective user canbe stored there which are then taken into account during conversion intoanimations and interactions.

The additional function of analysis and interpretation of a facialexpression, of a speech or text input of a user is advantageous inparticular if, in addition to communication via the representatives, thetwo users communicate with each other in the usual way by text and/orspeech messages (e.g. via VoIP and/or Instant Messaging) or webcams.

In order to make possible a particularly simple and intuitive input ofcontrol commands to the representatives, the presentation of thepossibilities of animation and interaction of the representatives takesplace in a tabular overview. The tabular overview is applied interminals which provide the user with a graphic user interface. Thus,with the help of the graphically presented table which contains theavailable control commands in the form of small graphic symbols (“icons”or also “thumbnails”), the user can select an action which is to becarried out by a representative. The overview table can also be calledgrid, matrix or raster.

The tabular overview preferably has a fixed number of classes in whichthe possibilities of animation and interaction are collected and fromwhich they can be retrieved. Thus the tabular overview can consist of atwo-by-three matrix wherein each of the six fields of the matrix standsfor one of the six finally fixed classes. The animations in the sixclasses are particularly preferably collected into the areas “mood”,“love”, “topics”, “comment”, “aggression” and “events”.

If, moreover, the users are provided with a drawing function to makepossible a real-time transfer of a drawing by a user on his terminal tothe other user on his terminal, yet another type of communicationresults for the users. Using a drawing tool a user can produce a drawingon his graphic user interface. The other user and communications partnercan then track creation of the drawing in real time on his terminal.Thus information can also be sent which can be presented only withdifficulty in writing or via the representatives.

Furthermore it can be proposed to integrate into the views of the twocommunications users a mood display which shows the present respectivemood of the two representatives. This mood display can be accomplishedin the form of a mood bar and/or in the form of a face laughing to agreater or lesser extent depending on the mood (“smiley”). Thus eachuser can directly see precisely what his own mood and that of theforeign representative looks like. The respective mood display can varyin the course of the animation of the representatives and as aconsequence of the exchange.

If additionally an automatic animation of a representative also takesplace in reaction to a change in the mood display, the behaviour of therepresentative is particularly varied and true-to-life. Thus e.g. therepresentative of the first user can automatically start to jump for joyif its mood display has exceeded a specific limit value.

The mood display can be presented in the most varied forms, e.g. even inthe form of a “thermometer” or a “temperature curve”. The current moodor humour of the user can also be displayed by colouring or otherwiseconfiguring the representative (combot). It is particularly advantageousif the depth of communications is also made to depend on the currentmood. For this, the system assesses the current mood display andmodifies the animations and/or interactions accordingly regarding therepresentative (combot) of this user.

The presentation of the two representatives at the first terminalpreferably is a mirror image or inverted mirror image of thepresentation of the two representatives at the second terminal. Thismeans e.g. that each user always sees his representative on the left andthe representative of the other user on the right. A clear allocation isthus guaranteed even if the representatives are identical.

Further particular advantages result if the following additionalfeatures are fulfilled:

The one animation of the at least one representative and/or theinteraction between the representatives preferably takes place dependingon predeterminable criteria, in particular criteria which are stored ina user profile which is allocated to at least one of the two users.

Moreover at least one of the two users can be provided with a selectionof animations and/or interactions to be transferred. This can also takeplace depending on predeterminable criteria, in particular criteriawhich are stored in a user profile which is allocated to at least one ofthe two users. A selection of animations and/or interactions to betransferred is proposed to at least this user.

In this context details relating to at least one of the two users, inparticular information regarding gender, age, nationality, mothertongue, speech habits or patterns, place of residence, interests and/orhobbies, can be predetermined as criteria.

It is also advantageous if one animation of the at least onerepresentative and/or the interaction between the representatives takesplace in response to a drag & drop command of a user, wherein the drag &drop command relates to the actual representative of this user or to therepresentative of the other user and wherein the animation orinteraction takes place depending on which of the two representativesthe drag & drop command relates to.

In connection with recognizing speech or text inputs or videorecognition, this can take place depending on predeterminable criteria,in particular criteria which are stored in a user profile that isallocated to at least one of the two users. It is advantageous if thepredeterminable criteria include details relating to at least one of thetwo users, in particular details regarding gender, age, nationality,mother tongue, speech habits or patterns, place of residence, interestsand/or hobbies.

The animation of at least one representative and/or the interactionbetween the representatives can depend on the mood display which, for atleast one of the two users, displays his current prevailing emotionalmood. Thus it can be provided in particular that the automatic reactionof a representative in response to a received emotion depends on thecurrent prevailing mood of the receiving representative. If, thus, e.g.the prevailing mood of a representative is well-disposed and thisrepresentative receives an aggressive emotion, the automatic reaction ofthe representative could be a simple shake of the head. However, if theprevailing mood of the representative receiving the aggression isnegative, instead of simply slightly shaking his head he couldautomatically clench his fist and swear.

Conversely the mood display which, for at least one of the two users,displays his current prevailing emotional mood can be modified dependingon the transferred emotion and/or interaction. Likewise the selection ofanimations and/or interactions to be transferred at least to one of thetwo users can be provided according to the mood display which, for atleast one of the two users, displays his current prevailing emotionalmood.

It is advantageous if the selection of animations and/or interactions tobe transferred is provided in the form of assembled groups and/orclasses. In this connection, the assembly of the classes and/or theselection of animations and/or interactions can take place automaticallyand pseudo-randomly controlled.

Finally, the present invention also comprises a system to carry out themethods according to the invention described above.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows an overall view of a user interface of a user for carryingout the method according to the invention;

FIGS. 2 a to 2 c show alternative embodiments of user interfacesaccording to the invention;

FIGS. 3 a to 3 f show a further alternative embodiment of a userinterface according to the invention;

FIGS. 4 a and 4 b show an example of an interaction between two virtualrepresentatives;

FIGS. 5 a and 5 b show two embodiments of the tables according to theinvention for selecting a control command;

FIG. 6 shows different control possibilities which are available to theuser with the help of the tables according to FIGS. 4 a and 4 b;

FIG. 7 shows the text recognition and interpretation (parsing) accordingto the invention;

FIGS. 8 a to 8 d show different processing possibilities of the controlcommands issued by the users;

FIG. 9 shows the respective inverted mirror view of both users;

FIGS. 10 to 28 show by way of example the progress of a communicationbetween two users using the method and system according to theinvention;

FIG. 29 shows an example of a complex communication with text-basedelements and with elements configured according to the invention.

DESCRIPTION OF PREFERRED EMBODIMENTS

Preferred embodiments of the present invention are now described by wayof example for a better understanding.

Below it is assumed that a first user named “Franz” is in communicationwith his friend “Manuela”, who represents the second user. The two arecommunicating over the Internet by means of their respective computers.Both Franz and Manuela have a communication application running on theirrespective computers for communication with each other. The userinterface 1 of this application is represented by way of example in FIG.1.

The user interface 1 is the interface which Franz uses to communicatewith Manuela. FIG. 1 thus reproduces what Franz sees on his screen whencommunicating with Manuela. On the screen of her computer Manuela has aninterface analogously structured to the user interface 1.

Below only the structure of the user interface 1 is described.

The user interface 1 is accomplished as an independent window with threemain sections 2, 3 and 4. Section 2 can be called animation section. Thevirtual representatives 5 and 6 of Franz and Manuela are presented inthis section. Section 3 is the text and control section. The textmessages exchanged between Franz and Manuela are presented here. Thecontrol panels for controlling communication are also accommodated insection 3. Finally, section 4 is the menu bar.

The two virtual representative 5 and 6 (combots) of both users are to beseen in the animation section 2. The virtual representative 5 of Franzis a car, while the virtual representative 6 of Manuela is a doll.Nametags 7 and 8 above serve to better allocate the representatives. Ascan be seen in FIG. 1 Franz's representative 5 is currently in animationphase and sending hearts 9 to Manuela's representative 6. In this wayFranz is expressing his affection for Manuela through his representative5.

Small windows 10 a and 10 b are arranged above the representatives 5 and6 in the respective corners. These windows show what actions are beingcarried out at this precise moment by the respective user. If e.g. apencil appears in window 10 b then Franz knows that Manuela is currentlyusing the drawing function described later in detail.

The text and control section 3 is divided into a messages area 11, acontrol bar 12 and a drafting area 13. The text messages alreadyexchanged between Franz and Manuela are to be seen in the messages area11. In order to compose and to send a text message to Manuela, Franzuses the drafting section 13. Franz can enter a text message to Manuelainto this section 13 by means of his keyboard. As soon as Franz hasproduced the text message, he can send it to Manuela by pressing thebutton 14. The sent speech message then appears both in Franz's messagesarea 11 and in Manuela's messages area.

In order to control his representative 5 Franz uses the control bar 12.The control bar 12 has several buttons 15. Different animations of therepresentatives 5 and 6 can be triggered by these buttons 15. Thus the“heart animation” indicated in FIG. 1 can be triggered by dragging anddropping the heart symbol onto Franz's car. By dragging the boxing gloveonto Manuela's doll, the representative 5 of Franz can thus be made topunch the doll.

An overview table with further control commands can be opened bypressing the button 16, as is presented by way of example in FIGS. 5 aand 5 b.

Button 17 makes possible the opening and closing (showing and hiding) ofthe animation section 2.

Moreover, via the button 18 presented as a pencil symbol, Franz can drawfree-hand in the messages section 11 any desired figures which areunderstood in real time in Manuela's messages section. An example ofsuch a drawn figure is emphasized by the reference number 19. Moreover,in the context of the section 11 the already-exchanged emotions are alsodisplayed symbolically, i.e. the emotions forming part of the history ofthis still ongoing communication are displayed. Here, e.g. an emotiongiven the reference number 19H, and presented as a “boxing glove” isdisplayed, a rather aggressive emotion which “Franz” had previously sentto “Manuela”. By clicking on the symbol 19H of this historic emotion itcan immediately be spontaneously repeated.

A history where appropriate of all past communications sessions can beretrieved via the menu bar 4 (by pressing the “history” button).Moreover it is also possible to access one's own files (by pressing the“files” button) in order, where appropriate, to send these to thecommunications partner. Finally, a session for joint surfing on theInternet can also be started via the button “surf*2”.

FIG. 2 a shows an alternative configuration of the animation section 2.Unlike FIG. 1, here section 2 additionally has mood displays 20.1 and20.2. Mood display 20.1 is a stylized face (a “smiley”) which by itsfacial expression illustrates the current mood of the respectiverepresentative and thus of the corresponding user. As can be seen,“Franz” is in a better mood than “Vroni”, as the laugh of the “Franz”mood display 20.1 a (smiley) is wider than that of the “Vroni” smiley20.1 b. The mood display can alternatively or additionally also beaccomplished in the form of mood bars 20.2 a and 20.2 b respectively.Here the length of the bar indicates the quality of the mood.

FIGS. 2 b and 2 c show, compared with FIG. 1, two further variants ofthe presentation of the interaction between two virtual representatives.The desktop of the user “Franz” is presented in each case.

In the case of FIG. 2 b the virtual representative (combot) 21 of hiscommunications partner “Vroni” is stored on the desktop 23. If “Franz”now wishes to send “Vroni” an emotion, he does so by clicking on themouse or dragging & dropping onto “Vroni”'s combot 21. In this case of aso-called “sent emotion” the presentation of a thought bubble 24 appearsimmediately on the desktop 23 of the sender (“Franz”) as FIG. 2 c shows.Both combots 22 and 21 and the transfer of the emotion itself here e.g.a flying heart which is flying from Franz's combot 22 to Vroni's combot21, are presented inside the thought bubble 24. Everything appears inmirror image on Vroni's desktop (not presented).

In the case of a “received emotion”, the following happens on thereceiver's desktop: initially, “Vroni”'s representative 21 is as a ruleno longer heeded by the receiver “Franz”. However, allocated to thestored representative 21, a thought or speech bubble 24 can suddenly andautomatically appear if the communications partner “Vroni” has sent acorresponding animation command from her computer to the computer of theuser “Franz”. The animation of the two representatives 21 and 22 thentakes place in the opened thought bubble 24.

If a user has stored several representatives of different communicationspartners on his desktop, the user can also direct a communication toseveral communications partners simultaneously. If the user e.g. wishesto send the same animation to two users simultaneously, he can combinethe two corresponding representatives into a group. The user can sendthe desired animation to both communications partners in a singleprocess by a single “drag & drop” onto the formed group. The most variedgroups can be created using this “intelligent” formation ofrepresentative groups, such as e.g. temporary or else permanent groups.Several individual representatives can also be combined into a singlegroup representative. The group representative (“Groupcombot”) is anindividual representative with whose help the user can enter equallyinto contact with a whole group of communications partners.

Additionally, the system provides the following reference or noticefunction: if the receiver “Franz” does not respond to this emotion bymanually reacting to it if the system does not cause an automaticreaction, a pointer 217 to this received emotion is displayed at Vroni'scombot 21. This pointer 21Z is e.g. the current number of emotions towhich there has yet been no response. Should the potential receiver“Franz” thus not be present for incoming emotions, he can subsequentlyrecognize immediately whether, and how many, emotions have arrivedduring his absence and can then react to them.

The system establishes whether the potential receiver has noticed ormissed the emotions on the basis of monitoring the receiver'sactivities. If e.g. the receiver performs no mouse or keyboard inputsduring the presentation of the emotion and for at least five secondsthereafter, the system assumes that the receiver has missed theanimation. Alternatively or additionally an activity recognition cantake place via a video camera which is connected to the receiver'scomputer. Using the camera, the system checks whether the receiver ispresent. The video camera can also be used by the receiver as an input.The receiver can then, e.g. by hand movements which are recognized bythe camera, send commands direct to his computer e.g. to control hiscombot or react to a received emotion.

In an additional step the camera can even recognize the user's bodylanguage, preferably in the form of a real-time monitoring. Thebehaviour of the user is constantly recorded via the camera. By means ofrecognition or interpretation software the system can interpret thebehaviour of the user and animate the virtual representative in realtime in tune with the user's behaviour. Thanks to the camera recording,it can e.g. be established that the user has just adopted an attitudethat indicates that he is sad. The user's combot will then automaticallysimultaneously express the user's sadness. Camera recognition allows thecombot to, as it were, imitate any behaviour of the user. The videocamera detects the mood or attitude or even the air of the user. Thedetected mood is then automatically transferred to the combot by thesystem. Thus if the user e.g. clenches a fist, this movement is recordedby the camera then interpreted by the system and finally causes theuser's combot to re-enact the user's movement: the combot clenches hisfist, just like the user.

A particularly intuitive communication can take place via the combotswith the just-described constant observation of the user. The user neednot issue his combot with active commands, but merely needs to sit athis computer and behave naturally. The user's unconscious, direct andintuitive reactions during the communication are transferred whollyautomatically directly onto the combot without the need for a consciousinitiative to that effect on the part of the user.

If a receiver has missed an animation, a pointer 21Z is displayed at thecorresponding combot. Additionally, an entry concerning the missedemotion is made in a logbook provided for the purpose (so-called“history”). The receiver can once more trigger or replay the missedanimation via the logbook and/or the pointer 21Z. Thus the systemprovides a type of recorder or memory function for missed animations.

A speech bubble is preferably presented in the case of a receivedemotion, but a thought bubble in the case of an outgoing emotion. Thedifferent manner of presentation can, however, relate to whether anemotion is already transferred or not. If a user only wishes to preparethe transfer of an emotion (editing mode and/or preview mode), a thoughtbubble appears on his desktop. Initially, nothing yet appears on thedesktop of the communications partner. However, as soon as the emotionis transferred (interaction mode) a speech bubble appears on bothdesktops. FIG. 2 c shows another variant:

In FIG. 2 c the two virtual representatives 21 and 22 are stored on therespective desktop 23 in FIG. 2 c. The animation takes place here bycombining the two representatives in an overall presentation, aso-called “arena” which preferably has the form of a tube or else acylinder 25.

FIGS. 3 a to 3 f illustrate a further variant of the operation andinteraction of the virtual representatives. FIG. 3 a shows the desktop,i.e. the screen surface of a user named Bob. On his desktop Bob hasstored a representative 59 in the form of a snowman. The representative59 is allocated to Bob's friend Alice. Bob can communicate with Alicevia the representative 59.

If Bob now wishes to communicate with Alice, he simply moves his mousecursor 41 onto the representative 59. As soon as the cursor 41 is overthe representative 59 (so-called “MouseOver”) a circular menu 60automatically appears which surrounds the representative 59 (see FIG. 3b). By clicking on a menu section, Bob can now trigger various actions.Alternatively, the menu display and selection can take place such thatBob clicks on the representative 59 so that the menu appears, keepingthe mouse button depressed in the process, then when the mouse button ispressed, moves the cursor 41 onto the corresponding menu point andfinally selects the menu point by releasing the mouse button (so-called“release”).

If Bob e.g. now activates the “message” menu section he reaches anapplication with which he can produce a text message for Alice. If Bobselects the “emotions” menu section with his cursor 41 (see FIG. 3 c) anoverview table 28 appears (see FIG. 3 d) as is to be seen in detail alsoin FIGS. 5 a and 5 b.

Numerous icons are listed in the overview table 28. Bob can place one ofthese icons on Alice's representative 59 by clicking, dragging anddropping (drag & drop). This is presented by way of example in FIG. 3 e.Bob drags an “angry smiley” onto the representative 59 in order to thuslet Alice know that he is in a bad mood. Once Bob has dropped the smileya further representative 61 (see FIG. 3 f) automatically appears onBob's desktop. The representative 61 represents Bob and displays theemotion selected by Bob. As soon as the animation of Bob'srepresentative is completed, it disappears again from Bob's desktop.

The animation selected by Bob also manifests itself on Alice's desktopsuch that the representative stored there which stands for Bob carriesout the animation selected by Bob. Alice's representative does notappear on Alice's side.

The FIGS. 4 a and 4 b show by way of example a typical interactionbetween two virtual representatives 26 and 27. The representative on theleft 26 (“little man” combot) has been animated by his owner, byselection of the animation command “bomb”, to throw a bomb at therepresentative 27 (“car” combot) of the communications partner. As aconsequence the virtual representative 27 is hit by the bomb andexplodes (see FIG. 4 b). The user behind the representative 26 may e.g.have selected the action “throw bomb” in order to express his angerabout the communications partner opposite.

FIGS. 5 a and 5 b illustrate two embodiments 28 a and 28 b of thecommand table which can be called up by pressing button 16 (see FIG. 1).

All the actions which can be carried out by means of a virtualrepresentative (combot) are presented in table 28 a in an overall grid29. Each available animation is presented by a corresponding square iconin the table 28 a. The icons can each be allocated according to commongroups (e.g. according to “love”, “friendship”, etc.). The overall grid29 is divided into two sections 30 a and 30 b. The basic animations(“basic emotions”) which are freely available to each user, such aslaughing, crying etc., are located in the first section 30 a. On theother hand special emotions (“gold emotions”) which are peculiar to eachuser are located in the second section 30 b. These idiosyncrasies of therepresentatives can be acquired by a user e.g. bought, exchanged ortraded with other users.

It is also provided that in the starting table 29 (overview table) anicon not only stands directly for an emotion or animation, butrepresentatively for a whole group of animations. Thus the heart icon 32stands for the group of “love animations”. By pressing the icon 32 afurther subtable 31 is opened from which the user can then select thedesired love animation for his representative. A group thus comprisesseveral variants of a basic presentation of an emotion, such as e.g. theheart presentation described here.

Those animations which cannot be allocated to a group are presented in aseparate column 33.

In the embodiment according to FIG. 5 b another type of distribution ofthe emotions is shown, wherein the overview table 28 b is limited to sixfields. Each of these fields stands for a whole class of animations. Therespective class (e.g. class 34 “mood”) is shown by pressing thecorresponding field in table 28 b. The desired animation can then beselected within the class. Another class comprises e.g. all types ofaggressive emotions and is symbolized in the starting table 28 b by abomb. The subtable which contains various types of emotions forselection opens by clicking on this symbol. The emotions collectedwithin a class differ not only with regard to their form ofpresentation, but also fundamentally. This means that various types ofemotions can be allocated to a class. They have a common meaning,statement of content or prevailing mood. The aggressive emotions classdescribed here comprises e.g. a bomb animation, a lightning animation ora shooting animation.

FIG. 6 illustrates how the desired animation is selected and triggeredby a user with the help of a table 28. There are essentially threevariants A to C, wherein the first two variants are carried out usingthe “drag & drop” principle. The three variants are indicated bycorresponding arrows.

In variant A the user drags the selected icon onto the correspondingrepresentative and drops it there. The thus-operated representative thenimmediately carries out the desired animation. In the example accordingto FIG. 6 a thundercloud is selected and dragged onto the foreignrepresentative 6. The consequence of this is that a thundercloud is sentfrom the actual representative 5 of the user to the foreignrepresentative 6 and soaks him.

In variant B the icon is dragged into the messages area 11 and droppedthere. This leads to the selected icon appearing in the messages area ofthe respective communications partner. By clicking on the icon thecommunications partner can then trigger the animation sent by thecounterpart.

In variant C the icon is simply clicked on by the user. The icon isthereby integrated into the drafting area 13 at the cursor positioncurrent there. Upon integration of the icon a suitable text canadditionally automatically be offered to the user. If the user thus e.g.clicks on the “birthday cake” icon the “Happy birthday!” text can alsoappear above the “birthday cake” in the drafting area 13.

By pressing the send button 14 the written text message is sent with theintegrated icon to the communications partner.

In FIG. 6 a small face which is also called “emoticon”, is also to beseen in the messages area 11. Such faces, which express a specific mood,can be inserted into the message text as shown. For this, the user needonly input the character string of the emoticon desired by him, e.g.:-), when writing a text message in the drafting area 13. This characterstring is then automatically converted into the corresponding face, here

. Upon operation of the send button 14 the text complete with emoticonis then sent to the communications partner.

Each individual emotion from the selection of emotions displayed intable 28 can also be immediately activated by double-clicking.

Automatic text recognition and interpretation (“parsing”) is presentedin FIG. 7. When a user inputs a text 35 into his drafting area 13, itssense is automatically ascertained. The ascertained terms are thenpresented to the user, here in the form of a speech bubble 36.Simultaneously, two animations very suited to the sense of thejust-inputted text are proposed to the user in the form of the icons 37.In the example according to FIG. 7 the user has inputted a greetingstext with birthday wishes. Accordingly, a “love animation” and a“birthday cake animation” are proposed to the user. It is also providedthat the animation of the representative in response to the sense of theinputted text is automatic, without the possibility of selection by theuser.

Various alternatives for processing the animation commands issued by theuser are illustrated in FIGS. 8 a to 8 d.

With the alternative according to FIG. 8 a an animation 38 a of therepresentative is immediately interrupted and replaced by a newanimation 38 b when the user issues his representative with a command tocarry out the new animation 38 b. There is a direct and delay-freeimplementation during this processing of the control commands, so thatthe behaviour of the representative has a rapid and dynamic effect.

Unlike FIG. 8 a, in the alternative according to FIG. 8 b the animationis completed first before the new animation 38 b takes place. Theoriginally proposed following animation 38 c is suppressed.

With the alternative according to FIG. 8 c all the animations triggeredby the users are executed in linear succession. There is no suppressionof animations. The requested animations are also placed according totheir chronological order into a so-called “playlist” and carried outsuccessively.

It is illustrated in FIG. 8 d how a repeated interaction between tworepresentatives can be processed and reproduced. A first user has hisrepresentative carry out an action 38 a. This is then interrupted by areplica of the representative of the second user 39 a, which for itspart is carried out instead until the representative of the first useragain reacts through the action 38 b.

The animation sections 2 a and 2 b of a first and second user arepresented mirror-inverted in relation to one another in FIG. 9. Thefirst user and second user employ their animation sections 2 a and 2 brespectively in the manner described in order to exchange emotions witheach other via their representatives. The exchange takes place over thenetwork 40 (e.g. the Internet). For both the first user (“my PC”) andthe second user (“your PC”) the user's own representative is presentedon the left and the foreign representative on the right, so that amirror-image view results. When communicating via the representativesboth users see the same sequence simultaneously in their respectiveanimation sections. Thus it could be said that both users see“everything”, i.e. the totality of the process in progress.

For the rest, it is also provided that virtual representatives can bebought, collected and exchanged by users. Thus some representatives mayexist only in limited editions or even be unique, so that differentrepresentatives have a different commercial value. The sale anddissemination of the virtual representatives can be substantiallyimproved by this measure.

FIGS. 10 to 28 show an example of a communication such as could takeplace between Franz and Manuela. FIGS. 10 to 28 each reproduce snapshots(“screenshots”) of Franz's user interface 1. FIG. 10 represents thestart of the communication and FIG. 28 the end.

In order to start a communications session with Manuela, Franz operatesthe E button 17 with his mouse cursor 41 (see FIG. 10). The animationsection 2 is thereby opened, in which the virtual representatives 5 and6 of Franz and Manuela are presented (see FIG. 11). Manuela's nametag 8is greyed-out, which means that Manuela is not yet in contact withFranz, i.e. Manuela is still “offline”. In FIG. 12 Manuela is now“online”, because Manuela's nametag 8 is now highlighted just likeFranz's. Moreover, a spotlight 42 is now likewise trained on Manuela'srepresentative.

As can be seen in FIG. 13, Franz has sent Manuela a first text message,to which Manuela also immediately replies (see FIG. 14). While Manuelais inputting her text, a hand appears in Franz's window 10 b, whichindicates that Manuela is carrying out an action right now. Subsequentto her text input Manuela indicates a “sad face” 19 with the alreadydescribed drawing function. Using the pencil presented in the window 10b, Franz can see that Manuela is drawing right now (see FIG. 15).

In response to Manuela's slightly mocking drawing 19, Franz inputs afurther text and moves his mouse cursor 41 onto the animation button 43,which presents a “boxing glove” (see FIG. 16). By dragging & dropping,Franz moves the boxing glove 43 onto Manuela's representative (see FIG.17), so that an interaction is triggered in which Franz's representativefires off a boxing glove at Manuela's representative (see FIG. 18).Manuela's representative is hit by this and falls over (see FIG. 19).Thereupon Manuela for her part triggers an interaction in which herrepresentative places a thundercloud above Franz's representative (seeFIGS. 20 to 22). In order to put this right again, Manuela thensubsequently sends puckered lips to Franz's representative via herrepresentative (see FIG. 23).

In order to express his good feelings about the puckered lips, Franzthis time moves his mouse cursor 41 onto the heart 44 (see FIG. 24) anddrags this onto his representative (see FIG. 25). An animation isthereby triggered in which Franz's representative sends out littlehearts (see FIG. 26).

Finally, Franz prepares yet another text message in his draft section13. He embellishes this with a closing greeting 45 drawn free-hand bymeans of the drawing function (see FIG. 27). To send the text message,Franz clicks on the send button 14 with his mouse cursor 41 (see FIG.28). After dispatch the message is presented in both Franz's andManuela's respective messages sections.

Another user interface 50 is presented in FIG. 29. This has analternative layout with the following areas:

Firstly, there is a communications area 51 which contains a messagessection and via which the current communication takes place in real timeor near-real time, and there is a preparation area 53 with a draftingsection in which the respective user can prepare his intendedcontributions (text, graphics, emotions etc.), before sending them tothe other user by pressing the send button. A slider 52 with control baris also provided which separates the areas 51 and 53 from one anotherand provides control elements for text input, for drawing etc. Thus thestructure of this interface 50 essentially also corresponds to theinterface already presented in FIG. 1.

Here in FIG. 29 an overview area 55 with history is now also provided inwhich all previous communications are listed. Listing can bechronological, thematic or user-related. Also located at the bottom endis an area 55 with a menu bar which contains various function buttonscomparable with the menu bar presented in FIG. 1.

The layout shown in FIG. 29 also has yet another navigation area 56which serves for navigation within a single (still ongoing or alreadycompleted) communication. For this there is i.a. a movable window withsegment 51 which includes a sub-area in the navigation area, whereinthis sub-area is then presented enlarged in the area 51. This is thus anenlargement or magnifying function. During an ongoing communication thesegment 57 always tracks the area 51 in real time. Through this“tracking” the user always has the overview and orientation within thecommunication. By moving the segment 57 he can jump at any time to anypoints which are then displayed enlarged in the area 51, so that theuser if appropriate can supplement the communication precisely at thispoint. Thus an interleaved supplementing and where appropriatemodification of a communication is made possible.

The user interface 50 also has another separate area in which therepresentatives 5 and 6 (combots) of the two users (here “Franz”'s carand “Vroni's” eye) are presented in interaction. In this case, however,not only the non-verbal communication is presented by emotions, as hasalready been described before (FIGS. 1-29). Here, the remainder of thecommunication which takes place between the two users is also nowdisplayed, such as e.g. transfer of files (file transfer) or text (bye-mail, SMS), instant messaging or chat as well as telephoneconversations (VoIP, PSTN) etc. For this an appropriate symbol 58 isanimated and presented, such as e.g. a document flying from combot 5 tocombot 6 which indicates a file transfer. Thus the users obtain totaloverview of all types of communication taking place between them.

Two or more users can communicate with one another in a particularlyattractive, versatile and varied way with the just-describedcommunications method and system. In particular through the use ofvirtual representatives and their animation or interaction, moods,feelings and emotions can be exchanged in a particularly effective andclear way between the users.

A very convenient non-verbal communication can be carried out with thedescribed invention, in particular through the user-friendly operationby mouse clicks and dragging & dropping. The described representatives(combots) and the presentation of the transfer of emotions that takesplace between them makes a very neat impression on the users and thus avery clear and direct transfer of the respective emotion, which can evenreproduce gestures, body language and facial expressions. Theinteraction between the combots, in particular the predeterminable andautomatically controllable interaction, offers the communicationspartners involved a novel form of communication, wherein the actuallevels of the communication of content combines with a playful level.The personal idiosyncracies and preferences of the users are taken intoaccount by system-supported recording and assessment of user-specificdata, and increase the convenience and acceptance of the communicationaccording to the invention.

1-17. (canceled)
 18. Method of telecommunication between a first userand a second user over a telecommunications network, the methodcomprising: providing a first connection between the first user and atelecommunications network via a first terminal; providing a secondconnection between the second user and the telecommunications networkvia a second terminal; allocating a first virtual representative to thefirst user and a second virtual representative to the second user;presenting the first and second virtual representatives on the firstterminal and on the second terminal; transferring information from thefirst user to the second user and vice versa by an animation of at leastone of the first and second representatives and by an interactionbetween the first and second representatives, wherein at least one ofthe animation and the interaction takes place in response to a drag &drop command of a user, and wherein an animation of the first virtualrepresentative takes place in response to a command of the first userand an animation of the second virtual representative takes place inresponse to a command of the second user.
 19. The method as recited inclaim 18, wherein the animation and/or interaction is presentedsimultaneously, parallel and in real time on the first and secondterminals.
 20. The method as recited in claim 18, further comprising atleast one of: directly interrupting an ongoing animation or interactionin response to a new user command to carry out a desired animation orinteraction; concluding an ongoing animation or interaction andpresenting a desired animation or interaction in response to a usercommand to carry out the desired animation or interaction; placing adesired animation or interaction in a waiting list of respectiveanimations or interactions to be carried out in response to a usercommand to carry out the desired animation or interaction; andinterrupting a first animation or interaction triggered by the firstuser and replacement of the first animation or interaction by a secondanimation or interaction triggered by the second user and vice versa.21. The method as recited in claim 18, further comprising: recognizingat least one of a speech and a text input by the first or second userinto the respective one of the first and second terminals; analyzing andinterpreting the speech or text input;
 22. The method as recited inclaim 21, further comprising: performing a video recognition of at leastone of the first and second user's facial expression; and analyzing andinterpreting the facial expression.
 23. The method as recited in claim22, further comprising: providing a plurality of suitable animation orinteraction possibilities in tune with a sense of at least one of thespeech input, the text input and the facial expression.
 24. The methodas recited in claim 23, further comprising: animating at least one ofthe first representative, the second representative and an interactionbetween the first and second representatives in tune with a sense of atleast one of the speech input, the text input and the facial expression.25. The method as recited in claim 18, further comprising: presentinganimation and interaction possibilities of the first and secondrepresentatives in a tabular overview, wherein the tabular overview hasa fixed number of classes in which the animation and interactionpossibilities are collected and can be retrieved.
 26. The method asrecited in claim 18, further comprising providing a drawing function soas to enable a real-time transfer of a drawing by at least one of thefirst and second users on a respective one of the first and secondterminals to the other one of the first and second users on the otherone of the first and second terminals.
 27. The method as recited inclaim 18, further comprising: presenting a mood display on a respectiveone of the first and second terminals indicating a current respectivemood of one of the first and second representatives;
 28. The method asrecited in claim 27, further comprising animating the representative asa reaction to a modification of the mood display.
 29. The method asrecited in claim 18, wherein the presentation of the first and secondrepresentatives at the first terminal is one of a mirror image and aninverted mirror image of the presentation of the first and secondrepresentative at the second terminal.
 30. The method as recited inclaim 29, wherein at least one of the animation of the first or secondrepresentative and the interaction between the first and secondrepresentatives takes place depending on predeterminable criteria. 31.The method as recited in claim 30, wherein the criteria are stored in auser profile allocated to at least one of the first and second users.32. The method as recited in claim 18, further comprising providing aselection of animations and/or interactions to be transferred to atleast one of the first and second two users.
 33. The method as recitedin claim 32, further comprising proposing the selection to betransferred to predeterminable criteria stored in a user profileallocated to at least one of the first and second users.
 34. The methodas recited in claim 33, the predeterminable criteria include detailsabout at least one of the first and second users.
 35. The method asrecited in claim 34, wherein the details include information relating toat least one of a gender, age, nationality, mother tongue, speech habit,speech pattern, place of residence, interest and hobby.
 36. The methodas recited in claim 18, wherein the drag & drop command relates to theat least one of the first and second representative, and wherein theanimation or interaction takes place depending on which of the tworepresentatives the drag & drop command relates to.
 37. The method asrecited in claim 22, wherein the recognition of the speech or text inputor the video recognition takes place according to predeterminablecriteria stored in a user profile allocated to at least one of the firstand second users.
 38. The method as recited in claim 37, wherein thepredeterminable criteria comprise details about at least one of thefirst and second users.
 39. The method as recited in claim 38, whereinthe details include information relating to at least one of a gender,age, nationality, mother tongue, speech habit, speech pattern, place ofresidence, interest and hobby.
 40. The method as recited in claim 27,wherein the and/or the interaction depends on the mood display, whereinthe mood display displays a current prevailing emotional mood of atleast one of the first and second users.
 41. The method as recited inclaim 27, wherein the mood display for at least one of the first andsecond users displays a respective current prevailing emotional mood,and wherein further comprising modifying the mood display according to atransferred emotion and/or interaction.
 42. The method as recited inclaim 32, wherein the selection is provided according to a mood displaywhich, for at least one of the first and second users, displays acurrent prevailing emotional mood of the respective user.
 43. The methodas recited in claim 32, wherein the selection is provided in a form ofassembled groups and/or classes, at least one of the assembly of theclasses and the selection of the animations and/or interactions isautomatic and pseudo-randomly controlled.
 44. A system of carrying outthe method as recited in claim 18.